Method and system for simulating surgical procedures

ABSTRACT

A system and method for converting static/still medical images of a particular patient into dynamic and interactive images interacting with medical tools including medical devices by coupling a model of tissue dynamics and tool characteristics to the patient specific imagery for simulating a medical procedure in an accurate and dynamic manner by coupling a model of tissue dynamics to patient specific imagery for simulating cerebral aneurysm clipping surgery.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a national stage application of PCT internationalapplication PCT/US12/31514 filed on Mar. 30, 2012, which claims thebenefit of U.S. Provisional Application Ser. No. 61/469,152 which wasfiled on Mar. 30, 2011 both of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

This application relates generally to a system and method for simulatingsurgical procedures. More specifically, this application relates to asystem and method for converting static/still medical images intodynamic and interactive images interacting with medical tools (such as,e.g., surgical tools, probes, and/or implantable medical devices) bycoupling a model of tissue dynamics to patient specific imagery forsimulating cerebral aneurysm clipping surgery.

“Medical errors kill as many as 98,000 people annually at a totalnational cost of between $37 to $50 billion for adverse events andbetween $17 to $29 billion for preventable adverse events.” “Surgicalerrors are the leading medical error” Source: To Err Is Human: Buildinga Safer Health System, Institute of Medicine. National Academy ofSciences. (1999).

Furthermore, out of 19,034 treated cerebral aneurysm cases in U.S.non-government hospitals between 2002-2006, 10,719 (56%) were treated bycerebral aneurysm clipping surgery. Even with the advent of endovasculartechniques, the more complicated aneurysms still require microsurgicalclipping.

During the course of high risk surgeries, such as, cerebral aneurysmrepair surgeries, for example, the absolute orientation of the braintissues is significantly altered as a surgeon pushes and cuts tissues toapproach the aneurysm area. Therefore, the current utilization of theadvanced surgery preparation and aiding systems such as Image Guided andNavigation Systems which are based on pre-registered 3D imageries, arelimited in assisting the surgeons. Also, surgeries, such as aneurysmrepair, are extremely time-sensitive, for example, due to variousprocedures including temporary vessel clamping to the aneurysm area.Therefore, the efficiency of the procedure is highly critical anddetailed planning based on the patient specific local geometry andphysical properties of the aneurysm are fundamental. To achieve a newlevel of pre-surgery preparation, 3D CT and MRI images are beingincreasingly utilized. However, those images offer only minor benefits,standing alone, for surgery rehearsal.

Surgeons lack a rehearsal and preparation tool that would provide themwith a realistic visual model with physical tissue properties.Currently, there is no capability for pre-surgery preparation thatallows a neurosurgeon to plan and physically rehearse the microsurgicalstrategy based on the patient-specific anatomy of the aneurysm. Hence,it is desired to have a “full immersion” surgical tool that encompasses:(i) realistic “life-like” 3D display of the patient-specific area ofsurgery (such as supporting cerebral aneurysm clipping surgery); (ii)modeling of the local patient-specific area of surgery geometry andphysical properties; (iii) interface enabling manipulation of thepatient-specific area of surgery model and virtually perform surgicalactions such as cutting, shifting and clamping; and (iv) interface toprovide feedback cues to the surgeon.

SUMMARY OF THE INVENTION

The disclosed system and method, called the “Cerebral Aneurysm SurgeryRehearsal Platform” (CA-SRP), provides a platform that allows aneurosurgeon to plan and physically rehearse the microsurgical strategybased on the patient-specific anatomy of the aneurysm. As a uniquesurgery preparation system, the CA-SRP provides patient-specific: (i)accurate modeling of the tissue mechanical properties; (ii) realistic 3Dimagery of the tissues (as seen in open/classic surgery); and, (iii)real-time, surgery-like manipulation of the dynamic and interactive 3Dtissue models (v) dynamic and interactive modeling of surgery toolsincluding aneurysm clips, implants, and other devices, integrated intothe 3 dimensional dynamic and interactive patient specific case.Accordingly, the CA-SRP provides the following clinical benefits for thepatient, the surgeon, and the hospital: (i) reduced potential forobjectively assessed intra-operative errors during microsurgicalclipping; (ii) improved potential for neurosurgeon's insightful andsuccessful response to adverse events; and, (iii) decreased operativetime.

Provided is a modeling system for performing a surgical simulation,comprising: a database for storing patient tissue image information thatare taken from, or derived from, medical images of a particular patient;the database also for storing standard characteristics of the tissue; adisplay; an image generator for generating a dynamic 3D image of tissuesof the particular patient for display on the display, the generatingutilizing the patient image information such that the dynamic 3D imageof tissues realistically represents corresponding actual tissues of theparticular patient; a tool interface for connecting to a real surgicaltool adapted for use with the modeling system; a user tool generator forgenerating a tool model of a user tool for dynamically interacting withthe 3D image of tissues via manipulations provided by a user; and a userinterface for the user to adjust parameters of the modeled tool, whereinthe tool model is displayed on the display dynamically interacting withthe 3D image of tissues according to the adjusted modeling parametersfor realistically simulating the medical procedure.

Also provided is a modeling system for performing a simulation of ananeurysm clipping surgery, comprising: a database for storing patienttissue image information that are taken from, or derived from, medicalimages of a particular patient, the image information including ananeurysm; the database also for storing standard characteristics of thetissue; a display; an image generator for generating a dynamic 3D imageof tissues of the particular patient for display on the display, thegenerating utilizing the patient image information such that the dynamic3D image of tissues realistically represents corresponding actualtissues of the particular patient and includes at least one blood vesselshowing an image of the aneurysm in 3D; a memory for storing a libraryof a plurality of models of different aneurysm clips to the user; a userinterface for selecting one aneurysm clip model from the plurality ofmodels for use as the aneurysm clip model for dynamically interactingwith the image of the aneurysm; a tool interface for connecting to areal aneurysm clip applier adapted for use with the modeling system; aninterface for the user to adjust the mechanical properties of the imageof the aneurysm; and a user tool generator for generating the selectedaneurysm clip for dynamically interacting with the image of the aneurysmvia manipulations of the real aneurysm clip applier by a user, whereinthe aneurysm clip is displayed on the display dynamically interactingwith the image of the aneurysm based on the mechanical properties of theimage as adjusted by the user for realistically simulating the aneurysmclipping surgery.

Also provided are additional embodiments of the invention, some, but notall of which, are described hereinbelow in more detail as exampleembodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the examples of the present inventiondescribed herein will become apparent to those skilled in the art towhich the present invention relates upon reading the followingdescription, with reference to the accompanying drawings, in which:

FIG. 1 provides a high-level schematic of an example Surgical Theater;

FIG. 1A provides another high-level schematic of a hardwareimplementation of the example Surgical Theater;

FIG. 2 a high-level diagram of an example of the Collaborative Theaterconcept;

FIG. 3 shows an example breakdown of a distributed simulation networkconcept for the example Surgical Theater embodiments;

FIG. 4 is a block diagram block diagram showing design level andpreliminary software design requirements for the Surgical Theater;

FIG. 5 provides an example high-level Realistic Image Generator (RIG)platform for the Surgical Theater;

FIG. 6 provides an example high-level architecture and workflow of aSurgery Rehearsal Platform (SRP) for the Surgical Theater;

FIGS. 7-8 are example screen shots of windows in a DICOM Volume Viewerexample for the Surgical Theater;

FIG. 9 shows a high-level block diagram of an example embodiment of theCerebral Aneurysm Surgery Rehearsal Platform (CA-SRP);

FIGS. 10A-10C are example screen-shots of an example rendered 3D tissuemodel of the CA-SRP interface; and

FIG. 11 shows example aneurysm clip model dimensions;

FIG. 12 shows an example real Aneurysm Clip Applier modified tointerface with the CA-SRP;

FIG. 13 shows a real Aneurysm Clip Applier that is connected to thesystem through a tracking, monitoring, and control device/interfacealong with simulation display for the example CA-SRP;

FIG. 14 shows a screen shot of an example aneurysm clip model libraryand simulation display for the example CA-SRP;

FIG. 15 shows an example 3D simulation display of an aneurysm andaneurysm clip for the example CA-SRP;

FIG. 15 shows a screen shot of another example aneurysm clip modellibrary and simulation display for the example CA-SRP;

FIG. 17 shows an example display of aneurysm clip modification for theexample CA-SRP; and

FIG. 18 shows a screen shot of a surgeon interface for performingmeasurements of an aneurysm neck before and after applying the clip forthe example CA-SRP.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

As a Commanding General of the US Army Medical Research and Command,General Lester Martinez-Lopez declared, “I look forward to the day whensimulation can be leveraged to its fullest extent to reduce medicalerrors, improve medical skills, and, above all, improve medical care forpatients”. The importance of the General's vision is emphasized in thesobering medical statistics: “Medical errors kill as many as 98,000people annually at a total National cost of ˜$37 to $50 billion foradverse events and ˜$17 to $29 billion for preventable adverse events”.From the perspective of a neurosurgeon Dr. El-Kadi M.D., Ph.D. explains,“Even the most routine surgical procedure has the potential forcomplications”. He further reflects: “the most difficult part for me ishow I prepare the case in my mind. For me, going to surgery is likegoing to war. You must identify your mission, be very well prepared,flawlessly execute your plan, and then return home safe. If you're notprepared, then you should not go”. Indeed, the highest level ofsurgeon's preparedness is required for performing high-risk surgeries,(e.g. cerebral aneurysm repair) where crucial decisions are made undersevere time constraints, which are dictated by the biological importanceand fragility of the brain and surrounding structures. In other words,the surgeon's preparedness is essential to avoid potentially devastatingconsequences and maximize the success of the clinical outcome. Thiscreates a critical need to increase the potential for a successfulsurgical outcome through avoidance of objectively assessedintra-operative errors and tactical refinement of patient-specificsurgical procedures, particularly in the field of neurosurgery. Thus,providing the neurosurgeon with the ability to iteratively plan andphysically rehearse the course of a particular neurosurgery without riskto the patient will substantially reduce surgical errors therebyincreasing the likelihood of a successful surgical outcome.

Currently, 3D CT/MRI scans are increasingly utilized in an effort toachieve a better level of surgical preparation, (6-11). However, duringthe course of a brain surgery, the absolute orientation of brain tissuesis significantly altered as the surgeons cut, retract, and dissecttissues. Present utilization of “advanced” surgery aiding systems, whichare based on pre-registered static 3D images, provide limitedpre-surgery assistance to surgeons since these systems are not capableof projecting the changes in tissue orientation and do not provide anytactical rehearsal capabilities. Previous research in commercial andmilitary aviation has clearly demonstrated that rehearsal prior to taskperformance can result in significant improved outcomes (12; 13).Unfortunately, as stated above, these pre-mission (or pre-surgery)simulation and training tools have been noticeably absent in healthcare.The CA-SRP allows neurosurgeons to develop, rehearse and refine thetactical strategies for patient-specific cerebral aneurysm clippingsurgery. Major complications that can result from this procedureinclude, among others, premature aneurysm rupture, postsurgical strokeor “re-bleeding” from improper clip placement. Furthermore, as certainphases of the aneurysm repair are extremely time-sensitive (e.g.temporary vessel occlusion of afferent vessels), the efficiency of theprocedure is highly critical. This emphasizes the importance fordetailed planning and the need for tactical rehearsal capabilities basedon the local geometry and physical properties of the patient-specificaneurysm.

The disclosed system and method (hereinafter the “Cerebral AneurysmSurgery Rehearsal Platform” (CA-SRP)) utilizes an improvement of theSurgical Theater system disclosed in U.S. patent application Ser. No.12/907,285 that was filed on Oct. 19, 2010, and is hereby incorporatedherein by reference. The Surgical Theater is a computerized system thatconvents medical images into dynamic and interactive images by couplinga model of a specific tissue's dynamic attributes to patient specificimagery. This dynamic and interactive image/model creates a new, novel,and original standard for medical imagery and has many applications.Among others, the system/method can be utilized by medical imagerynavigation systems and image guided and robotic surgery systems that canenhance their planning performances by utilization of Surgical Theaterdynamic and interactive tissue models coupled with the patient specificimagery.

The Surgical Theater address challenges in surgeries that involvehigh-risk activities, such as heart and brain surgeries including valverepair and replacement, bypass surgeries, brain tumor removal, Aneurysmclipping, and others.

For example, in the case of open/classic brain surgeries, such as braintumor and brain aneurysm, for example, the Surgical Theater converts CTand MRI images and creates a realistic three dimensional (3-D) model ofthe brain tumor or aneurysm area with a dynamic modeling of theorganisms, including the tumor, along with the surrounding tissue andblood vessels. The system is connected to life-like surgery tools,responding to actions taken by the surgeon, helping him/her to betterprepare for the surgery. The Surgical Theater system simulates realisticevents such as brain swelling, damage to blood vessels, brain tissueshifting during an operation blocking access to the remaining parts ofthe tumor, among others. The system can be used as a planning andpreparation tool, allowing surgeons to tailor a specific surgicalstrategy for a given case, maximizing the surgery efficiency whileminimizing the risk.

Surgical Theater System Overview

FIG. 1 provides an example embodiment for one application of the system1 where a patient specific scan image (CT, MRI or similar) (14) is fedto the system's console (10), an algorithm that creates a 3 dimensionalrealistic anatomy display (18) adds texture, shadow, shadowing and othercues to the image, a mechanical properties algorithm (16) assignsmechanical behavior characteristics to the image and transfer the imagefrom static/still image to a dynamic and interactive image/model.Interfaces with or without force feedback (20) are connected to thesystem allowing the surgeon/operator (12) to manipulate the image/modelthat the system creates; the surgeon can select tools and implants fromlibraries of tools and implants including characteristics of those toolsand implants. The surgeon then performs a virtual surgery on amanipulateable, dynamic and interactive 3 dimensional image/model of hispatient organism in a realistic and dynamic manner.

The system includes an executive program that runs and manages all thesystem components and updates the status of the sub components accordingto the surgeon/operator (12) actions. For example, when the surgeon usesthe interface (20) to push a tissue (such as by using a chose tool) thathe sees in the display (18), the mechanical properties model (16)receives the information regarding the force that was applied, e.g., thedirection of force; the tool that is being used including its materialand shape and other mechanical characteristics of the tool, then themechanical properties are used to calculate a new state of the 3dimensional orientation an ad setup of the image according the forcethat was applied, the executive program send the calculated 3dimensional matrix to the realistic anatomy display (18) that wascreated by the mechanical properties algorithm (16), the realisticanatomy display calculates the new image and its cues due to the changeof image e.g., a new set of shadows and shadowing due to the neworientation of the image components are determined. Simultaneously, themechanical properties model (16) send a set of parameters to the forcefeedback interface (20), these parameters include information of theforce that the surgeon/operator (12) needs to sense due to theinteraction with the organs (the force that the organ returns after thesurgeon pushes or otherwise interacts with the tissues). This process ofcalculation of new stage at each one of the system's components (14, 16,18, 20) is executed rapidly and continuously in cyclic manner, and eachcycle is completed within a frame time of milliseconds, allowing thesurgeon/operator to receive real-time and realistic cues and real-timereactions to his actions.

The Surgical Theater is a system, as shown in FIG. 1A, that integratesseveral computers (PCs) 2A-2 n, one or more databases 3A-3 n and otherhardware components (e.g., networks 5, 6) and proprietary software intoone complete system 1 (see both FIGS. 1 and 1A) that is structured intoan immersive chamber/console sized about as big as a small walk incloset (see console 10 in FIG. 1). Once the surgeon 12 starts thesystem, the surgeon loads the set-up parameters of his patients whichinclude details of the patient to allow the system to up-load therelevant data, the Surgical Theater than loads all the patient'savailable CT and MRI images from a patient images 14 into thedatabase(s) 3 and other information that concern the simulated modelssuch as patient age, gender and so on (some or all of which may beobtained from external entities 8, such as medical databases, forexample). The system utilizes tissue information parameters 16 from asystem database. The system 1 performs a segmentation process andidentified the Entities of the organ, Entities are vessels, tissue,tumor, and so on to create the simulated image model 18 shown to thesurgeon on the display of the device. The system provides realistictactical feedback 20 via feedback mechanisms to add further realism tothe simulation.

The system applies the layers of the realistic visual, the mechanicalproperties and other relevant parameters 16 from the system database(s)and characteristics relevant to the case, all applied on the top of theCT and MRI images 14 from the patient images database(s) 3 andsynchronized with those images. The synchronization creates, forexample, vessel mechanical properties that are ‘clamped’ or ‘attached’to the vessel images and so on to provide realistic simulationcapability. The surgeon can be provided the ability to “fine tune” themodels and adjust the mechanical properties of a certain area of theorgan. For example, the surgeon may adjust the elasticity and othermechanical characteristics of the Entities behavior.

Subsequently, after such a set-up, the Surgical Theater projects the 3dimensional organ model 18 presented in a realistic visual fidelity withrealistic features such as; texture, shadowing and other features thatadds realism to the simulated image. Each segment of the visual model 18is coordinated and corresponds with an appropriate mechanical propertiesmodel from the system database 16 and other relevant properties of thespecific case.

At this stage, the system allows the surgeon to browse and chooses fromthe system's virtual libraries 16 in the system database the relevantsurgery tools and other elements (in the system software terms thosetools and elements are “Entities” as well) that he may need to performthe surgery (or other procedure). Such elements may include; seizers andclamps, clips for aneurysm, artificial heart valves, and other elementsappropriate for the specific case. (Adding additional systems 1′, 1″ . .. connected to the system 1 via a network 9—such as over the Internet ora private network—can result in a collaborative theater platform,described in more detail later in this disclosure.)

All of the various Entities are represented by the system inhigh-fidelity distributed models and functioning in a distributedarchitecture, e.g., each Entity typically has a separate subEntity,where the subEntity is, for example, a “visual entity” or “mechanicalentity” and so on. Each subEntity exists in one of the differentenvironments (e.g., the visual system environment, the mechanicalmodeling environment and so on, described in more detail below)distributed among a plurality of computers. Each such subEntity isresponsible for its own performance (i.e. presenting the realisticvisual of the Entity, or performing the Entity's mechanical operation,for example).

The subEntities communicate via a distributed network (described in moredetail below) to synchronize and coordinate the subEntities into a oneintegrated Entity compound model. For example, when a tissue is beingpressed by a surgery tool, the surgery tool pressure characteristics(e.g., the location, orientation and amount of pressure and so on) isdistributed via the network, each one of the subEntities is responsiblefor ‘listening’ and concluding if it is being affected by this surgerytoll pressure; once a subEntity determines that it is being affected,each such subEntity (for example, tissue Entity) models the affect ontheir subEntity model, e.g., the visual subEntity, presents the visualeffects (such as bloodiness of the tissue), and the mechanicalproperties models the shift of the tissue. Each subEntity distributesthe change—for example, the tissue location and dimension changes—overthe network so the other subEntities will be able to determine if theyare being affected by this change. At the end of such action, all thesubEntities of the tissue for the above example, (and the otherEntities), become accustomed to, and, if needed, adapt their states andthe models to, the new action that was sourced and initiated, in theabove example, by the surgery tool.

Thus, the various functions (subEntities) can be distributed amongvarious computers connected in a peer-to-peer network utilizingdistributed data and state duplication (for keeping local copies of thestate of the simulation), all listening on the network for any actionthat impacts their portion of the simulation, in which case they updatetheir parameters via the network to keep the system accurate, which may,of course, impact other functions in other subEntities, which willtherefore catch that fact by their monitoring of the network, leading tofurther updates, and so on. In this way, the system distributes thefunctionality among many computers in a parallel fashion so thatupdating can occur much quicker than it could if only a single computerwere used. Only those subEntities impacted by a change need respond, andthus network traffic can be reduced to essentials.

The Surgical Theater allows the surgeon to record his actions and savethem for later playback, to demonstrate his surgery plan to the chiefsurgeon or resident, or, to share information with other surgeons,demonstrate new techniques he is working on, practice the surgery, andso on. The system's interfaces to the surgeon includes surgeryinterfaces (e.g., seizers handles) that include force feedback that isdelivered to those tools to allow the surgeon to sense the forcefeedback cue of his actions, realistically simulating an actualprocedure.

Once the surgery tools and the other Entities are selected by thesurgeon, they are integrated into the virtual surgery scene and turninto an integrated element of the simulated scenario including realisticvisuals features and mechanical properties and operation propertiesfeatures that are applied to each one of those selected items. Forexample, the simulated scissors reflect mechanical characteristics ofreal scissors and will cut in the simulation as the real scissors do,and, aneurysm clips, when placed at the simulated vessel, simulatesblocking the blood flow.

Next, the surgeon performs the surgery actions at any stage of thevirtual surgery; the surgeon can “freeze” the simulation and rotate theorgan to observe the area of his interest from different orientationsand perspectives. The surgeon can “mark point of time” of the virtualsurgery and can command a “return to the mark point”. For example, thesurgeon can mark the time before clamping an aneurysm and return to thispoint of time while “un-doing” all the actions that took place afterthis point of time. In this fashion, the surgeon can evaluate differentsurgery approaches of a selected phase of the surgery without restartingthe entire surgery from the original starting point. Several such ‘markpoints’ are available allowing the surgeon to return and “re-do” actionsand exams/rehearse on several selected phases of the surgery. SurgicalTheater use may include surgeon rehearsals toward a surgery; surgeondemonstration to the chief surgeon or resident; surgical practice anddevelopment, testing, and validation of tools and methods, and knowledgesharing.

Collaborative Theater

FIG. 2 shows a high-level example implementation of the CollaborativeTheater concept that was introduced with the Surgical Theater. Byleveraging next generation broadband infrastructure 25, individualsusing SRPs 21, 22, 23 . . . from different hospitals will be connectedallowing surgeons across the nation and across the globe tocollaboratively plan a surgery case, e.g., surgeons from two or moredistributed sites step into their SRP and rehearse, together, on apatient case toward a surgery. This Collaborative Theater allowssurgeons to study the best practice methods by observing previousSurgical Theater cases as well as providing remote teaching andmentoring. The Collaborative Theater allows all the hospitals that areconnected and using the SRP to gain access to the up to date accruedknowledge and most recent “best practices”.

System Level Design

The system level design description is outlined in the precedingsections. The visual rendering engines analyze 3D MRI and CTpatient-specific images and create computerized segmented modules thatrepresents the anatomical structures and features of the particularimage. The medical market has a vast number of advanced Digital Imagingand Communication in Medicine—DICOM (1) viewers. Their feature setsrange from layered black and white slices in 3 different panels thatcould be cross-referenced to a complete ability to fly through staticsubsets of 3D images of patient's organs. In addition, there are 4D and5D features that record various functional and dynamic changes of organsin a form of a movie clip. As magnificent as those captured images ormoving sequences might be, they are a fixed set of snapshots images intime.

The Surgical Theater takes existing 3D conversion processes and adds thefeatures specific to the human tissues and structures based on physicaland mechanical properties that are then stored in the system database.Once this patient-based model is set in motion in the virtual world, theSurgical Theater introduces a set of virtual surgical tools that allowthe surgeon to manipulate (push, cut, clamp, etc.) those models similarto real surgery tissue manipulation, providing an intuitive experiencefor the surgeon.

FIG. 3 provides a breakdown of an example Surgical Theater distributedsimulation network (Surgical Theater DIS (ST-DIS) is presented). Each ofthe components (i.e., blocks) in the figure is an isolated computationstation (that can be executed on a stand-alone computer or collection ofcomputers) with a designated set of functions. The stations areappropriately connected with a regular support network 31 (such as anEthernet network, for example) that handles slow irregular traffic, liketransferring of vast amounts of DICOM data. Upon more intense dataprocessing demand, the stations are supported by a specialized

Distributed Interactive Simulation (ST-DIS) Network 32 that is ahardware isolated network used only for high priority simulation data(which can be implemented in high-bandwidth Ethernet, for example). TheST-DIS Network 32 carries volatile simulation information and allows forsuch an exquisite simulation load distribution.

The Surgical Theater's ST-DIS is a network architecture for buildinglarge-scale virtual worlds from a set of independent simulator nodes.The simulator nodes 33-38 are linked by the networks and communicate viaa common network protocol (such as TCP/IP, for example). The ST-DISinfrastructure enables various simulators to interoperate in a time andspace coherent environment. In the Surgical Theater's ST-DIS ST-DISsystem, the virtual world is modeled as a set of “Entities” thatinteract with each other by means of events that they cause. Thesimulator nodes 33-38 each independently simulate the activities of oneor more of the Entities in the virtual world of the simulation andreport their attributes and actions of interest to other simulator nodesvia messages on the network. The other simulator nodes on the networkare responsible for “listening” to the network messages, determiningwhich ones are of interest to them (based on the Entities they aresimulating) and responding appropriately.

One of the features of the ST-DIS network and simulation architectureconcerning distributed interactive simulation is that there need be nocentral server or processor. Each simulation application maintains itsown copy of a common virtual environment in its own memory or database.Representations of this environment are distributed by various means toall simulation applications prior to any real time operation. ST-DIS isbasically a peer-to-peer architecture, in which data is transmittedavailable to all simulators where it can be rejected or accepteddepending on the receivers' needs. By eliminating a central serverthrough which all messages pass, ST-DIS reduces the time lag for asimulator to send important information to another simulator. This timelag, known as latency, can seriously reduce the realism, and thereforethe effectiveness, of a networked simulator. Effective distributedsimulation depends on very low latency between the time that a newstate/event occurs for a simulated entity to the time that thestate/event is perceived by another entity that must react to it. Anydelay introduced by the training device could result in negativereinforcement to the trainee.

Referring again to FIG. 3, the Archive Server 34 is generally used toperform the tasks of downloading and retaining in a database largeamounts of data necessary for simulation.

In addition, the Archive Server 34 can be used to prepare obtained datafor further use in the simulation. Note that because its duties aretypically global in nature, and not critical to the simulation activity,the Archive Server 34 is typically only connected to the support network31.

FIG. 3 shows a network architecture that includes an off line “support”network (31) that “Archive Server” (34) that loads the medical images(CT/MRI) and additional initialization data stored in a database (forexample, the patient name, age and so on and files to be included in thescenarios such as surgery tools libraries) “Debrief Server” (38) thatrecords control inputs and store the scenarios and all the actions in atimeline information and allows playback of scenarios and actions. Thereal time network (32) is the network that transfers messages betweenthe systems node during the simulation in a real time fusion—one way forimplementing this network can be a Distributed Interactive Simulation(DIS) network (32), the components that connected to this network are;Control Input (33) that connected to the surgeon/operator systemsinterfaces, this node has an optional direct physical connection to theHost Computer (35) that may be implemented in a case that the real timerequirements of the system cannot be satisfied by the DSI network and adirect physical connection between those node sis needed. The HostComputer (35) includes the executive manger program and other models andsimulation components and it is responsible for the real timesynchronization and timing of the entire systems.

The Theaters Initialization Systems (TIS) (36) performs that systemallocation and setup for each one of the nodes, for example, when thesurgeon select a specific tool to use, the TIS allocates/activates theappropriate models of this tool for generating an accurate toolsimulation (with tool characteristics stored in a database) for all thenodes assuring that all the nodes are set up with the sameinitialization. The Image Generator (36) performs the rendering andvisualization tasks of the scenarios. The Host Computer (35), the TIS(36), the Image Generator (36) and the Debrief Server receive andexchange information with off line for initialization from the Supportnetwork (31) and receive and exchange information with the real timenetwork (32) for “on line” and real time simulation.

Needed organ surface and volume data are extracted from an existingMRI/CT scan stored in the database. To obtain 3D organ surface data, thesystem can use a DICOM viewer and data management system such as theOsiriX (or comparable) that is open source software implemented forApple Macintosh computers, for example. By “tapping into” OsiriX'sability to generate 3D surfaces of organs and organ groups based on thevoxel density values with Objective C source code, the Surgical Theateradds an ability to store information about the 3D surfaces and organtypes that describe into a flat file in a database. The entire set ofparts of this study stored in this manner in the system database so thatit is later transferred to the Image Generator Station 37 that recreatesthe patient-specific images based on standard characteristics of theorgans. Once the necessary rendering data is obtained, the renderingplatform for Image Generator Station 37 is applied to the image. Forthis, a proprietary Image Generator algorithm is integrated (such as aFlight IG; see the features in the separate headings for the RealisticImage Generator—RIG) with a Visualization Tool Kit.

The IG has unique features that deliver fine cues such as shadowing,texture, and material properties that are assigned to the visual modelsand as further detailed in the RIG sections. Not only does the IG createrealistic and fully immersed environments by using those features, itcan also process large volume of visual data base models under hard realtime constraints. Enabled by the combination of the DIS architecture andthe “Entity” design, the network traffic is minimized and the anatomy ofthe peer-to-peer nodes create a highly efficient real time system.

After the patient-specific images have been successfully rendered,various physics libraries are added in order to create propersimulation. Pushing and manipulation of the brain tissue is simulatedusing extensive research embodied in modeling platforms such as theOpenTissue (or comparable) collections of libraries that are available.The OpenTissue, for example, is an open source collection of librariesthat models volumetric shells and other complex behavior of3-dimensional shapes. Customized libraries can also be developed foruse. Specificity of the brain tissue physics and mechanics propertiesthat derived from the research of mechanical properties of brain tissuein tension can be utilized, for example. Experimental papers areavailable that provide mathematical models of the mechanicalmanipulation of animal brain samples. Dynamic and realistic interactionof simulated surgical tools with the simulated tissues are implementedin the algorithms and approaches as described by Viet H Q H, Kamada T,and Tanaka H T, An algorithm for cutting 3D surface meshes and/orvolumetric models, 18th International Conference on Pattern Recognition,4, 762-765. 2006 (incorporated herein by reference). The work looks atvarious tools and tissue types to create a realistic simulationspecifically for implementation of surgical simulations.

The software code of the example Surgical Theater is written in acommercial environment such as C++, with the code being designed to runin windows operating system, a Linux system, or compatible. In thecoding development process, emphasis is given for the code real timeexecution and code efficiency all aimed to maintain a real time systemperformance while minimizing the latencies.

The visual system driver located in the Image Generator (37) is designedwith an optimizers environment, such as OpenGL or similar, enableshigh-performance rendering and interaction with large models whilemaintaining the high model fidelity demanded, providing attention todetail while maintaining high performance in a cross-platformenvironment.

For computing efficiency purposes, each of the visual model's Entitieshave several Level of Details (LOD) representations; high LOD ispresented in areas of the simulation scene in which the surgeon needshigh resolution at, and, lower LOD is presented in areas of thesimulation scene in which the surgeon has no immediate interest orinteraction with. For example, tissue visual model is presented in highLOD in the area around the surgeon interaction and with lower LOD inareas that the surgeon doesn't have immediate interaction with. The LODcan be dynamically adapted: a surgeon's actions such as pointing thesurgery instruments toward a specific area can be utilized by the LODoptimization algorithm for the dynamic allocation of the LOD forspecific section of the visual models.

The typical system's computer is a PC with a multiple core (multipleprocessors) which provides flexibility and growth potential. Thecomputer system includes random access memory, Ethernet ports, systemdisk, and data disk.

For the validation of the Surgical Theater (image quality, realism,image controller and manipulation), the skills and experience of seniorsurgeons are utilized. The surgeons are used to evaluate the system byperforming specific surgical procedure while comparing it against theirvast neurosurgical experience as well as against a specific case thatthey have already operated and is being simulated in the SurgicalTheater.

The Surgical Theater Block Diagram of FIG. 4 describes the functionalityand the flow of the process (vs. the actual network connection of FIG.3) from the row data of the scanted image DICOM 41 through the processof segmenting the row data (to identify soft tissue, vessels and so on).Then the Image Generator assign visual representation of each segment(shadow texture and so on), this image is connected via the DIA 44network to a projection interface 46 and to the Host 45 that will updatethe image generator 43 with the surgeon actions that are connectedthrough the Surgeon Interface 47 and the mechanical Properties and othermodeling that the Host includes that all will reflect the new state thatthe Host will send to the IG 43 during each simulation cycle.

By eliminating the central server through which all messages pass,ST-DIS dramatically reduces the time lag for one simulator (computer) tosend important information to another simulator (computer). This timelag, known as latency, can, if too large, seriously reduce the realism,and therefore the effectiveness, of a networked simulator. Effectivedistributed simulation depends on very low latency between the times anew state/event occurs for a simulated entity to the time thestate/event is perceived by another entity that must react to it. Anydelay introduced by the training device results in the negativereinforcement to the operator (e.g., the surgeon).

According to the recommended practice for communications architecture(IEEE 1278.2), the underlying communications structure should support100 ms or less latency for packet exchange for closely coupledinteractions between simulated entities in real-time (e.g. simulatinghigh performance aircraft in a dogfight or simulating a surgeonperforming brain surgery). This requirement is based on human reactiontimes that have been the basis of Human-In-The-Loop (HITL) flightsimulator designs for many years.

Within the ST-DIS system, the virtual world is modeled as a set ofEntities (as described previously) that interact with each other bymeans of events that they cause. An Entity is a sub-component in thesimulated scenario, such as tissue, specific characteristics (suchas—tissue mechanical properties,) creating a sub group of that “tissueentity”. Another Entity can be a blood vessel, for example, and so on.Each Entity can have several subEntities that operate in a distributedmanner (such as on different simulators/computers). Together, thosesubEntities are combined to create the complete Entity model. ThosesubEntities are, for example: the Visual subEntity that holds andsimulates the Entity's visual feature and characteristics, or, theMechanical Properties subEntity that holds and simulates the Entity'smechanical feature and characteristics. Each of those subEntities modelcode can run in a different computer (or group of computers) such as aPC, and they communicate with each other as well as with other Entitiesvia the ST-DIS network. The simulator nodes, independently simulate theactivities of one or more Entities (or subEntities) in the virtual worldof the simulation and report their attributes and actions of interest toother simulator nodes via messages on the ST-DIS network. The othersimulator nodes on the network are responsible for “listening” to thenetwork messages, determining which ones are of interest to them (basedon the entities they are simulating) and responding appropriately.

The above-described Surgical Theater architecture is based on thisDistributed Simulation concept thereby enabling pioneer and exclusiveabilities to deliver a premier fidelity which is an essentialrequirement for creating immersive scenarios crucial for the rehearsingof open/classic surgeries where the surgeon(s) interacts with theorgan(s) by direct human sense. As each Entity is divided to itssub-components (visual, mechanical properties and so on), and as each ofthose subcomponents/Entities' simulation code runs in a separatecomputer, this can maximize the computation power, and by that thecreation of a unique and exclusive premier fidelity, fine cues, andcomputing capabilities while handling terabytes of information underhard “real-time” constraints while maintaining real time performance(e.g., less than 100 millisecond latency), the core capability of theFlight Simulation technology.

The Surgical Theater facilitated a visual rendering engine whichanalyzes 3D MRI and CT patient-specific images and creates computerizedsegmented modules that represents anatomical structures and features ofthe particular image. Medical market has a vast number of advanced DICOMviewers, but as magnificent as those captured images or moving sequencesmight be, they are based on a fixed set of snapshots in time. TheSurgical Theater takes existing 3D model conversion algorithms and addsthe features specific of the human tissues and strictures based onphysical and mechanical properties creating a “living” image with modelsthat reforms the patient specific CT/MRI images according to actionstaken by the surgeon and based on the models that simulate themechanical properties of each pixels in the image and realistic visualcharacteristics models. Once this patient-based model is set in motionin the virtual world, a set of virtual surgical tools (that can includeaneurysm clips and clip appliers, implants such as bone joint implants,or other devices) are introduced allowing the surgeon to manipulate(push, cut and etc.) those models similar to a real surgery tissuemanipulation. Thus, the Surgical Theater provides an intuitiveexperience for the user.

For the Image Generator, the Surgical Theater of the example embodimentintegrates a proprietary Flight Simulation Image Generator algorithmwith a visualization code such as Visualization Tool Kit (VTK). Asdetailed in the following sections, the Surgical Theater Realistic ImageGenerator has features that deliver fine cues such as shadowing,texture, and material properties that are assigned to the visual models.

The Realistic Visual Sub System

This section focuses on the “realistic visual” segment of the SurgicalTheater that is a modification of a Flight Simulation Image Generatorthat is capable of rendering satellite images into realistic 3dimensional images and models that are converted into the SurgicalTheater realistic Image Generator (RIG) handling and real time renderingCT/MRI DICOM images into a patients' specific realistic and dynamicCT/MRI images and models that are crucial for the open/classic surgerieswhere the surgeons interact with the organ by direct human sense.

The use of a visual system in the creation of the immersive simulationsystem in the field of Human factor Engineering is important; studiesdemonstrate that a high percentage of the immersion is constructed andcontributed by the level of fidelity and realism of the visual systemthat the operator (e.g., pilot or surgeon) interacts with. Findings showthat operators who rehearse on high fidelity visual systems completedthe memory task including self-report of confidence and awareness statesin significantly higher levels than the low fidelity group. Asignificant positive correlation between correct ‘remember’ and ‘know’responses, and in confidence scores, are found when utilizing highfidelity, realistic simulation.

As outlined above, the Surgical Theater creates a realistic “life-like”digital rendition of the surgical site and the surroundingtissues/structures. Since this digital rendition is patient-specific and“life-like”, it sets Surgical Theater apart from other simulators thatuse generic imagery to create approximate renditions of the surgicalsite, or, other system that simulates noninvasive procedures such asendoscopic, vascular and similar procedures, where the surgeon/operatorinterfaces the organism with a camera that has its own visualcharacteristics that are defined and limited by the camera specificationand are very different from the visual characteristics of the bare anddirect eyes view of the open/classic surgeon's where the surgeoninteracts with the organism with direct sense of his eyes However,realistic “life-like” rendering presents a surmountable task due to thecomplexity of the properties of the living biological tissues. In orderto create such high degree of realism, the Surgical Theater includes aReal Image Generator add-on (RIG): a visual system wherepatient-specific images of the surgical site, together with surroundingtissues, is realistically presented and can be manipulated in thisall-purpose manner.

FIG. 5 shows a RIG Architecture Block Diagram. Data Base box—collectionof the mesh modules based on the patient-specific CT/MRI, 3D andsegmented images, pre-processing of the images, smoothing, masking,scaling. Graphic Creator box—Interface to the graphics card. ST-DISInterface box—Interface to the ST-DIS network. The figure shows ahierarchy diagram of the visual systems. The system includes anexecutive program that runs and manages all the system components andupdates the statutes of the sub components according to thesurgeon/operator and the status of all the sub components as they areread through the DIS network (502). The Operating/Executive Engine (501)is responsible for the initialization of all the software and hardwarecomponents in a way that all the system's components are working withthe same data bases (for example, the set of tolls that the surgeonchoose). When the scenario starts, the Operating/Executive Engine (502)performs the cycle and timing control and perform the task of managingeach component to complete its calculation cycle within the time framethat it is planned on in a way that all the system's sub componentsreceive the information from the other sub components on a timely mannerallowing the overall system to complete the simulation cycle in a giventime frame. For example, when an action is taken by the surgeon andtransmitted by the DIS network (502), the Feature Generator (504) readsthe relevant part of this action/consequence of this action ascalculated by the mechanical properties algorithm, the Graphic Creator(503) change the image according to this action (for example, move avessels that was pushed by the surgeon), then calculates the changesthat need to be applied on the image as a result of this change, forexample, creating a shadow resulted by the change of the vessel locationand orientation. This cycle is executed rapidly and continuously managedby the Operating/Executive Engine (501) in a cyclic manner in a way thateach cycle is completed within a frame time of milliseconds allowing thesurgeon/operator to receive real time and realistic cues.

CA-SRP General Description:

The Surgical Theater concept can be adapted to support the CA-SRPProcess. CA-SRP converts a patient specific CT and MRI imageries andcreates a realistic three dimensional (3-D) model of the aneurysm areawith a dynamic modeling of the organisms, including the surroundingtissue and blood vessels. The CA-SRP is connected to real surgery tools(including any real surgical device that can be manipulated by thesurgeon and interfaced to the CA-SRP)in a way that the handles of theuser interface that the surgeon works within the CA-SRP are similar tothe ones that he holds in the surgery, responding to actions taken bythe surgeon, helping him/her to better prepare for the surgery. TheCA-SRP system simulates realistic events such as brain swelling, damageto blood vessels, brain tissue shifting during an operation that blocksthe surgeon's access to the area he planned to access, as well as othercomplications such as; (i) Inappropriate temporary clip placement willnot stop the flow of blood into the aneurysm, (ii) Improper managementof the aneurysm will result in bleeding if temporary clips were notplaced or not sufficiently placed (iii) If the temporary clips were onfor a long time a feedback of possible stroke will be generated (iv) Ifthe aneurysm clip will obstruct a vessel coming out of the aneurysm oris attaching at the aneurysm neck, a stroked patient outcome will bereported.

CA-SRP Cerebral Aneurysm Surgical Rehearsal Platform (CA-SRP) iscentered around a software simulation engine, based on state of the artalgorithms for segmentation, soft tissue and collision detectionalgorithms that calculate orientation interaction between all theelements and detects when an interaction occurs (such as the surgerytool touching a tissue or vessel). This interaction information(location, angels and so on) is populated to the models of the tissuefor calculating the tissue deformation due to this interaction a s wellas to the surgery tool to provide force feedback to the surgeon hand dueto this interaction. The area of interest (i.e. Aneurysm) is presentedto the surgeon on the workstation, using a high fidelity visual;real-time, real 3D (stereoscopic visual) and realistic (life likevisual—texture, shadowing, shininess) image generator that presents boththe organ as well as the surgery tools being used by the surgeon.

Aneurysm repair surgery is extremely time-sensitive due to variousprocedures such as temporary vessel clamping in which the blood flowtoward the aneurysm area is blocked. The time-efficiency of theprocedure is highly critical and detailed planning based on the local,patient specific geometry and physical properties are fundamental andwill result with an enhanced clinical outcome and a better operationalefficiency. Surgical Theater is allowing surgeons to obtain criticalinsights for refining the surgery strategy and enhancing the surgeryoutcomes.

CA-SRP Architecture:

The CA-SRP realistic behavior of deformable tissue modeling is utilizingFinite Element Methods (FEM) using mass lumping to produce a diagonalmass matrix that allows real time computation . Additionally, itutilizes an adaptive meshing that is necessary to provide sufficientdetail where required while minimizing unnecessary computation.

The CA-SRP's brain tissues modeling is based on published studies ofthis area, for example, studies may include the followings, yet, CA-SRPcan adopt any description of tissues' mechanical properties: i) Miller Kand Chinzei K. Mechanical properties of brain tissue in tension. JBiomech 35: 483-490, 2002, and, ii) Miller K, Chinzei K, Orssengo G andBednarz P. Mechanical properties of brain tissue in-vivo: experiment andcomputer simulation. J Biomech 33: 1369-1376, 2000, both incorporatedherein by reference.

CA-SRP design employs patient specific data, CT and MRI imageries thatare converted to a 3D soft tissue model using our above describedalgorithm. This algorithm uses image enhancing processing andsegmentation methods to create a high fidelity segmented model thatpresented to the surgeon and react to the surgeon actions. The 3D softtissue model is then handled in real time by our STDE (Soft TissueDeformation Engine) that uses tissue properties from the mechanicalproperties studies. Other inputs to the STDE include the User Interface(UI) tools used by the surgeon and the resulting actions on the model.The resulting deformed model is fed into a high fidelity image generatorthat displays the model in a realistic way to the surgeon.

The CA-SRP code runs in Windows platforms coded using standarddevelopment tools, utilizing available software packages such as .NET,WPF, OpenGL and available simulation frameworks such as SOFA and GiPSi.The information shared between the different modules of the simulation,is distributed using a proprietary protocol; the Surgical DistributedInteractive Simulation (SDIS) which is based on the Flight Simulation“Distributed Interactive Simulation—DIS” protocol, and which can beadapted from the Surgical Theater concept introduced above. The SDIScore idea is to minimize the amount of data needed to be shared andtherefore, allowing real-time performance with scalability andindependence of the different distributed system components that executeseparate portion of the simulation (i.e. visual, mechanical properties).

SDIS Based Architecture:

The SDIS based architecture facilitates a unique and exclusive abilityfor premier fidelity, fine cues and computing capabilities whilehandling large volume of information under hard real-time constraintswhile maintaining real time performance which is the core capability ofthe Flight Simulation technology. One of the features of the SDISnetwork is that there is no central server or processor, each simulationnode (nodes may be: Image Generator, User Interface, Mechanical Modelingcomputer and so on) maintains its own copy of the common virtualenvironment—vessels, tissues and other models that are held andmaintained at each of the simulation node; each such model is handles asa separate “Entity”. This architecture enables several PCs to worktogether in a synchronized manner under hard real time constraintsallowing CA-SRP's pioneering and unique capabilities to deliver apremier fidelity of the simulated scene. This creates an immersivescenario that allows rehearsal of open/classic surgeries where thesurgeons interact with the organ by direct human sense.

Once the surgery tools and the other Entities are selected by thesurgeon, they are integrated into the virtual surgery scene and turninto an integrated element of the simulated scenario including realisticvisuals features and mechanical properties and operation propertiesfeatures that are applied to each one of those selected items, forexample—the scissors have the real mechanical characteristics and willcut as the real scissors do, and, Aneurysm clips, when placed at thevessel, blocks the blood flow.

The CA-SRP system as is compose by the following units or combination ofsub parts of the units depended on the configuration, volume that needsto be simulated and the specific application. These are similar to thosefor the Surgical Theater system as shown in FIG. 4, but modified asdescribed in this section. The sup components can run in Severalseparated Computing Processor Units in multiple PCs (FIG. 9):

The workstation that the surgeon works on is the User Interface 101. TheImage Generator 102 operates similarly to the like device in theSurgical Theater. The Simulation Executive Manager 103—synchronizes thereal time operation of the system, runs, and executes the modelingprograms. The STDE Workstation 104—This PC handles the STDE (Soft TissueDeformation Engine). The Archive Server 105—This station holds all therelevant files and data and able to record the procedure for futuredebriefing and data collection, and this PC also serves as the networkdomain controller. The IOS (Instructor Operation Station) 106 is formonitoring and controlling the training session, also allowing theinstructor to “inject” events. Also serve as the “Master of Ceremony”and will activate the whole training session.

Each of these Computing Processor Units connects via the SDIS networkwith a network switch (not shown).

Scenarios of Operation

The surgeon uses the CA-SRP to virtually position the head of thepatient, to determine the exposure of scalp, craniotomy site and size,and brain retraction to achieve the desired vessel exposure in thecorridor 905 shown in FIG. 10A. This part is done in the using the UIsimulated operating tools as well as with computer mouse and keyboard.The CA-SRP covers the stages of craniotomy, dura opening, and splittingof the fissure the vessel exposure through temporary clamping throughthe Aneurysm clip application and removal of temporary clips.

Typical Scenario of Operation: Typical Initial Setup

The surgeon feeds the set-up parameters of his patient which includedetails of the patient that allow the system to up-load the relevantdata, the CA-SRP then loads the patient's CT, CTA, MRI and MRA imageriesand other information that concern the simulated models such as patientage, gender and so on.

3 dimensional imagery of a portion patient head is presented at 906 inFIG. 10A; The surgeon positions the head in a desired orientation at 906(Head Orientation).

Similarly to the actual surgery, the surgeon virtually creates thecorridor 905 to approach the aneurysm by virtually removing the tissuesall the way to the aneurysm site with the UI. The surgeon marks the areaof the site of interest/Aneurysms 904.

The system preferably automatically performs segmentation process andidentifies the “Entities”; Entities are vessels, tissues, and so on 903,although manual segmentation can also be provided for. This is doneefficiently because of the arrangement of hardware and software thatutilizes the parallel processing features of the system, greatlyimproving system performance in contrast with manual simulations.

“Start Simulation” is initiated: The brain tissues that were “removed”in the initialization process virtually retract back to expose the areacontaining the abnormal vessel formation. The system then applies, atthe marked area of the site of interest/Aneurysms, the layers of thetissues mechanical properties and the realistic visual, and load otherrelevant parameters and characteristics relevant to the case, allapplied on the top of the CT and MRI and synchronized with thoseimageries. The synchronization creates, for example, vessel mechanicalproperties that are ‘attached’ to the vessel imageries. Surgery tools(which may include aneurysm clips, surgical implants, and other devices)libraries are included in the virtual scenario and available for thesurgeon.

The surgeon has the ability to “fine tune” the models and adjust themodeling parameters of the simulation by changing the mechanicalproperties of a certain area of the organ; for example, the surgeon mayadjust the elasticity and other mechanical characteristics of theEntities behavior to better match the mechanical behavior of the tissuebased on his experience and his anticipation, such as at the Angledshape Aneurysm Clip 902 and the Aneurysm Clip Applier 901 to impact tooland tissue interactions.

Consequently, the CA-SRP efficiently projects the 3 dimensional organmodel presented in a realistic visual fidelity with realism features of:texture, shadowing and other features that adds realism to the originalimage. Each segment of the visual model is coordinated and correspondingwith appropriate mechanical properties models.

FIG. 10B shows additional views of the simulation, further showing ananeurysm clip applier 912 after it was used to apply a straight clip 911on a vessel, and an angled clip 910 on a vessel.

Typical Scenarios of Use:

The surgeon browses and chooses from the system's virtual libraries therelevant surgery tools and other elements (in the system software termsthose elements are “Entities” as well) those Entities may include;scissors, bi-polar electro-cautery, suction tips, and clamps, clipapplier as well as a variety of Aneurysm clips 930, as shown in FIG. 14,which also shows a patient specific aneurysm model 933 based on thespecific patient's medical imagery and the selected clip 932 in thepatient specific environment.

Similarly to the actual surgery, the surgeon virtually dissects theaneurysm away from the tissue and the feeding vessels and exposes theneck to receive the clip/vessels and tissue retract or extract per theappropriate mechanical properties models and surgeon's actions thesurgeon also selects temporary clip from Clips library; the clip isrealistically modeled and can be handled by the surgery tools.

Also similarly to the actual surgery, the surgeon virtually places atemporary clip on the feeding vessels in order to isolate it from thenormal circulation, and the surgeon selects a clip from Clips library toaddress various positions, shapes, and sizes of Aneurysms (the libraryof clips includes the commercial clips', sizes/shapes/lengths of clips).The selected clip model appears in the scenario; the clip is an accurateand realistic model of the real clip and can be handled by the surgerytools.

Further similar to the actual surgery, the surgeon then virtually placesa clip across the neck of the aneurysm. The surgeon rotes the view andlook to observe and evaluate the clip placement, as shown in FIG. 15,showing the clip 940 applied on the aneurysm neck. The surgeon can thenremove the temporary clips, and puncture the aneurysm. If the aneurysm'sneck is not completely occluded, bleeding will occur.

As in real-life surgeries, inappropriate temporary clip placement willnot stop the flow of blood into the aneurysm, whereas appropriate clipwill cause the aneurysm to deflate or obliterate. Improper management ofthe aneurysm will result in bleeding if temporary clips were not placedor not sufficiently placed. FIG. 15 shows a situation where the aneurysmneck 941 is squeezed, shrinks and expands 942, indicating that a longerclip may be required. If the temporary clips were on for a long time afeedback of possible stroke will be generated. If the aneurysm clip willobstruct a vessel coming out of the aneurysm or is attaching at theaneurysm neck, a stroked patient outcome will be reported. At any stageof the virtual surgery; the surgeon can “freeze” the simulation androtate the organ to observe the area of his interest from differentorientations and perspectives.

The surgeon is able to “mark point of time” of the virtual surgery andthen can command a “return to the mark point”; for example, the surgeoncan mark the time before clamping the aneurysm and return to this pointof time while “un-doing” all the actions that took place after thispoint of time. In this fashion, the surgeon can evaluate differentsurgery approaches of a selected phase of the surgery without restartingthe entire surgery from the original starting point. Several such ‘markpoints’ will be available allowing the surgeon to return and “re-do”actions and exams/rehearse on several selected phases of the surgery.

CA-SRP will be mainly used as a planning and preparation tool toward apatient specific surgery. Thus, allowing surgeons to tailor a specificsurgical strategy for a given case, maximizing the surgery efficacywhile minimizing the risk all contributing for an enhanced surgeryoutcome.

Additional scenarios of the CA-SRP use may include: (1) Surgeonrehearsals toward a surgery;(2) Surgeon demonstration to the chiefsurgeon; (3) Surgeon peer review and collaborate with colleague that mayhave CA-SRP through internet or other network connectivity; (4) Surgeondemonstration to a resident; (5) Surgeon researching and developing newmethod; (6) Resident/fellow practice; (7) Scrub techs/nurses/physicianassistants practice will allow them to understand the surgeons role andhence their role in these operations; (8) Platform for development,testing and validation of surgery equipment, tools, or equipment; forexample—aneurysm clips, that will be exams in a realistic simulatedenvironment; (9) Surgeon community platform to share knowledge andaccrued experience; (10) Platform for resident and surgeon evaluationexams and certification; and (11) Platform to promote the use ofspecific surgical tool or instrument such aneurysm clip.

CA-SRP allows the surgeon to record his actions and save them for laterplayback and to demonstrate his surgery plan to the chief surgeon,resident, or, to share information with other surgeons or to demonstratenew techniques he is working on, and so on. CA-SRP's interfaces to thesurgeon includes surgery interfaces (i.e. handles) scissors handles,by-polars and bayonet forceps and aneurysm clip applier. Thoseinterfaces may or may not include force/haptic feedback that isdelivered to those tools to allow the surgeon to sense the forcefeedback cue of his actions. Providing force/haptic feedback can beespecially useful in providing a realistic simulation, and thus ispreferable.

Tailored, patient specific, design of cerebral aneurysm clip: The CA-SRPincludes a search engine that allows the surgeon to locate an aneurysmclip that matches the patient specific aneurysm and vessels 3dimensional geometrical structure. By feeding set of physical parameterssuch as; overall aneurysm clip length, determination of the aneurysmclip shape (number of angled shapes, sequence and angel values), numberof aneurysm clip holes (number of holes, location, sequence andshape/diameter) and other structural characteristics, as shown in FIG.11, items 913, 914, 915. Additionally, a graphic user interface allowsthe surgeon the option to work with a 3 dimensional model of an aneurysmclip and to design its own clip to match the patient specific aneurysmand vessels 3 dimensional geometrical structures—when the surgeoncompleted the design of the aneurysm clip (by either methods) the modelof aneurysm clip with the surgeon's design, appears in the CA-SRP. SeeFIG. 10C showing the Aneurysm Clip Applier 907 applying Straight shapeAneurysm Clip 908 The surgeon then can send the tailored design clip ina 3 dimensional photo file (jpeg, bitmap and comparable) for rapidmanufacturing. If a clip that matches the surgeon's input of the aboveparameters is commercially available, the CA-SRP provides the aneurysmclip manufacturer, part number and the 3 dimensional model of theaneurysm clip appears in the CA-SRP. The surgeon then uses the clipapplier to grab and hold the clip, to open the clip against the clipspring while the force of the clip is sensed in the surgeon's hands. Thesurgeon then maneuvers the clips and applies it, using the clip applier,on a vessels or the aneurysm. The surgeon can repeat this process asmany times as he desires with different clips.

This method and platform used to tailor a specific graft shape anddesign for bypass surgeries, a specific bolt screw or any other implantfor spine surgeries, orthopedic implants and many other applications.The platform also supports integration of any medical imagery withsurgery tools (such as clips, implants, and other surgical devices) anda user interface and tools for modifying the implants and allowingtailored design to match the implant to the patient specific case basedon a dynamic and interactive model.

Case Study Example—Scenario of Operation of a Tailored Made, PatientSpecific Design of an Aneurysm Clip:

The surgeon/operator starts with one of the available baseline clipmodels that the surgeon selects from a library 930, such as shown inFIG. 14. The surgeon/operator then evaluates the clip on dynamic andinteractive model of a patient specific (based on medical imagery suchas CT, MR, X-ray, Ultrasound and others) as shown in FIG. 15 anddiscussed above.

The aneurysm shape is changed due to the dynamic nature of themodeling—the clip pressure on the aneurysm tissue cusses squeeze andreshape/expansion of the aneurysm neck, as shown at 941, 9421 n casethat the evaluations on a patient specific subject determine that theselected clip is not sufficient (does not exclude the entire aneurysm orcreate stress on the surgeon gong vested and others . . . ) thesurgeon/operator can choose different clips for evaluations from alibrary 950, as shown in FIG. 16, where the surgeon is shown selecting anew curved clip 951 for applying to the patient specific aneurysm 952.The surgeon/operator can perform measurements of the aneurysm neckbefore and after applying the clip, as shown in FIG. 18, where ameasurement tool can either be attached to the cursors or be served asindependent/additional measurements tool 1001; the cursors are placed onthe patients' anatomical model for measurements 1002, for example, tomeasure the aneurysm's neck; the orientation of the measurement's viewis changed and a different observation angles is used 1003; andmeasurement results appear in a window and a X, Y, Z of each cursor ormeasuring tool is presented 1004.

The surgeon/operator can change length, shape, Angles and so on tocreate a tailored, specific clip design that is the best fit for thespecific patient, as shown in FIG. 17, with a modification from clip 960to clip 961. The surgeon/operator can define areas that he would want toget several options of clips (for example—clips with overall length oflength, 10, 15 and 17 millimeter or angle of curves of 15, 20 and 25degrees and so on). A 3D model is created for manufacturing of the clipas shown in FIG. 17. Hence, a tailored made clip designed specificallyto match the patient's own anatomy based on the simulation results canbe manufactured and sent to the surgeon to be used and to be applied inthe actual surgery. Similarly, other surgical tools and implants couldbe customized in a like manner, by first testing their operation viasimulation. This can be used to improve surgical results in a manner notavailable without simulating the surgery in advance.

In the patient specific modeled simulated environment that is built onthe foundation of the patient's medical imagery (CT, MR, X-ray,Ultrasound etc.) the surgeon: (1) creates the corridor by removing andshifting tissue to expose the targeted site (aneurysm, faulty vessel ,faulty heat valve, fractured bone, damaged knee, damaged hip, damagedshoulder and so on). (2)—the simulated environment is segmented andincludes: bone, soft tissue, vessels, nerves etc. Therefore the corridorthat is created is realistic and accurately represents the limitedapproach to the that will be available to the surgeon in real surgeryand it will take into consideration the anatomical obstacles (forexample, eye tunnel bone that block the approach to an aneurysm thatneeds to be removed). (3) when the site that needs to be treated isexposed, the limited available approach and the orientation of theapproach represent the realistic limited work environment that thesurgeon will face in the operating room. (4) the surgeon browse theSurgery Rehearsal Platform (SRP) implants library and select acommercially available implant to try or, a generic model—both areprovided with the option for modifications. (5) the surgeon remove anddissect any anatomical structure that needs to be replaced (i.e. faultyheart valve or faulty knee) (6) the surgeon maneuver the new implantthrough the corridor with the realistic limited space available. (7) thesurgeon places and attached/apply the implant on the treated site. (8)the simulated placement takes into consideration and it based on thepatient's anatomical structure such as; thickness of the bone availablefor attaching the implant (artificial knee, artificial shoulder,artificial hip etc.) and/or thickness of the vessel available forattaching the implant (bypass graft, artificial heart valve etc.), thedimension and orientation and the implant site etc. (9) a 3 dimensiondesign tool allows the surgeons to adjust modeling parameters of thesimulation to modify the implant (or other implantable device) to betterfit the patient's specific anatomy—for example, the length and angels ofan aneurysm clip can be modify in order to create a tailored designedclip that will best fit the patient's specific aneurysm's size and shapeand the specific orientation of the approach/corridor to the aneurysm.Other example may be modification of the dimension of the round or ovalheart valve to better match a patient's own anatomy. Additional examplemay be the tailored design of the thickness, shape and length of a platethat is placed to treat a broken bone. Other examples of adjustingmodeling parameters may be a tailored design of a graft to treat bypassor aneurysm, a tailored design and shape and angles of the artificialknee, artificial shoulder, and artificial hip. (10) the SRP and themodeled artificial implant support the performances of adjustments andalignments needed for the implant, for example aligning bands and stripsof a knee that being done in the operating room—the SRP allows theperformance of this alignments in advanced in the simulated environment.For example, the lengths of the bands and strips of a knee can bepre-detainment in the SRP/simulated environment before the surgerystart. (11) once the surgeon complete the design, he commend the SRP tocreate a “manufacturing model”—a file with the 3 dimension model withaccurate dimensions (length, thickness, angel and so on) all based onthe surgeon's tailored design. (12) instruction for pre-alignments iscreated, for example, instructions for alignments of the bands andstrips of a knee. (13) a file or a printout of the design is crateredand is sent to the implant manufacturer to be built (14) the SRP cansupport an inventory control of commercially available implant in a waythat the hospital purchase the specific implant only after it was triedand verified in the SRP, and only then being ordered from the vendor.

Applications for Image Guided Systems and Operating-Microscopes:

The Surgery Rehearsal Platform (SRP) includes a software module thatties the platform together with Image Guided systems andOperating-Microscopes. By superimposing and projecting the surgery planthat was created by the surgeon using the SRP on the top of thenavigation path or the operating-microscopes image, the surgeon is ableto follow his own planned actions; such as orientation of the surgerytools, the planned placement and approach of the aneurysm clips and soon.

By push of a button, the surgeon can see in the Image Guided systems orthe Operating-Microscopes screen the video clip of the actions that heplanes (i) the corridor entry location, corridor approach orientation(ii)the aneurysm clip placement, the aneurysm clip approach of placement, the aneurysm clip orientation (iii) the clip applier and other surgerytools placement, the clip applier and other surgery tools approach ofplacement, the clip applier and other surgery tools orientation.

This integrated/superimposed image allows surgeons to overcome blockedvisibility and hinder visibility challenges that the surgeons may haveduring the surgery due to the fact that the tools (such as clip applieror other device) may block the surgeons sight and view of themicrosurgery site, and thus prevent the surgeon from seeing the vesselor other part on which he operates. The integrated/superimposed imageallow the surgeon to repeat the actions that he planned with the SRP inthe actual surgery, while overcoming the visual and other challengesallowing a more efficient microsurgery procedure and superior guidanceprovided to the surgeon, based on his own plan.

Mechanical Integrating of Real Clip Applier

The system includes an interface to a real surgery tool, as shown inFIG. 13, the Real Clip Applier 919, that is connected to the systemthrough a tracking, monitoring, and Control device/interface 920. Thisdevice interface 920 monitors the real clip applier movements and 6degrees of freedom orientation of the applier (x, y, z and oiler anglespitch, roll, yaw). This interface 920 monitors the applier positing(i.e. close, open or any position in between) action. This interface 920is connected to the Clip Applier latching tips 916 via the—Clip Applierhandles 917, as shown in FIG. 12. Referring again to FIG. 13, theSimulated clip applier 918 is synchronized such that it follows andreplies to action taken by the Real Clip Applier 919 transmitted to thesimulator by the Tracking, Monitoring and Control device/interface 920.

In the process of the simulated surgery, the surgeon holds the Real ClipApplier 919 and performs the surgery actions, the Simulated clip applier918 follows, presents, and reflects the surgeon's actions in thesimulated environment by synchronized actions of the Simulated clipapplier 918 that follows the Real Clip Applier 919. The Real ClipApplier 919 is an example of the way that the system connects a realsurgery tool to the simulated environment; other surgery tools such as:CA-SRP's interfaces to the surgeon includes surgery interfaces,o-scissors, by-polars, and bayonet forceps among others. Thoseinterfaces may or may not include force/haptic feedback that isdelivered to those tools to allow the surgeon to sense the forcefeedback cue of his actions.

Clinical Challenges:

Identification and approach to the feeding vessels: Judgment for theoptimal placement and orientation of the aneurysm clip to exclude theaneurysms from the cerebral circulation while minimizing the stress onthe surrounding vessels.

Choosing specific clips from various clips available: All under severtime constraints. Surgical Theater is allowing surgeons to obtaincritical insights for refining the surgery strategy and enhancing thesurgery outcomes.

Clinical benefits: The CA-SRP a patient-specific, tailored to eachspecific case of each patient, preparation tool for surgery, used bysurgeons placed in the Neurological surgery department. It isanticipated that the CA-SRP will enhance the surgery outcomes byminimizing surgeries' critical time segments such as the phase of theAneurysm temporary vessel clamping. The CA-SRP provides an enhancedmeans to pre-plan the precise clamping orientation and the best courseof the approach to the Aneurysm area based on accurate rehearsal as wellas selecting the best clip in advance and based on the patient's ownanatomy. It is believed that this will result in the quality (efficacy)of the Aneurysm repair to be maximized while adverse events such asstrokes are minimized.

Specific clinical benefits: (i) Prior selection of aneurysm clips beforeeven entering the Operating Rom (ii) Prior plan of optimal approach tothe feeding vessels (iii) Surgeon can perform “what if” scenarios,evaluating different surgery strategies; different clip and differentapproach. (iv) The surgeon plans the optimal placement and orientationof the aneurysm clip to maximize the exclusion of the aneurysms from thecerebral circulation while minimizing the stress on the surroundingvessels resulting in—improved clinical outcome Enhanced surgeryefficiency and Better Outcome and Reduced possibility for adverseevents. Furthermore, the CA-SRP is a more than adequate platform forresidents and fellows in training and will address several “trainingrelated” challenges that our market study tracked. The major trainingchallenge that the CA-SRP will address is the challenge of trainingbrain surgeons with less available hands-on and Operation Room timeavailable due to the restricted duty hours;

Resident's Challenges:

“Residents must not be scheduled for more than 80 duty hours per week”**Source: ACGME, 2002 “Many programs have struggled to ensure residentsare receiving adequate training since July 2003”++Source: The NationCongress of Neurological Surgeons Washington Committee

The CA-SRP assists residents and fellows to gain more experience bybeing well prepared in an efficient manner to each patient-specificsurgery by utilizing the CA-SRP. The CA-SRP delivers the followingbenefits, both to the surgeons as well as to the residents and fellows:(1) Reduced surgical adverse events by rehearsing on patient specificsimulation with realistic immersive system; and (2) Increasingoperational efficiency by reducing surgical duration.

Many other example embodiments of the invention can be provided throughvarious combinations of the above described features. Although theinvention has been described hereinabove using specific examples andembodiments, it will be understood by those skilled in the art thatvarious alternatives may be used and equivalents may be substituted forelements and/or steps described herein, without necessarily deviatingfrom the intended scope of the invention. Modifications may be necessaryto adapt the invention to a particular situation or to particular needswithout departing from the intended scope of the invention. It isintended that the invention not be limited to the particularimplementations and embodiments described herein, but that the claims begiven their broadest reasonable interpretation to cover all novel andnon-obvious embodiments, literal or equivalent, disclosed or not,covered thereby.

What is claimed is:
 1. A modeling system for performing a surgicalsimulation, comprising: a database for storing patient tissue imageinformation that are taken from, or derived from, medical images of aparticular patient; said database also for storing standardcharacteristics of said tissue; a display; an image generator forgenerating a dynamic 3D image of tissues of the particular patient fordisplay on said display, said generating utilizing said patient imageinformation such that said dynamic 3D image of tissues realisticallyrepresents corresponding actual tissues of the particular patient; atool interface for connecting to a real surgical tool adapted for usewith the modeling system; a user tool generator for generating a toolmodel of a user tool for dynamically interacting with said 3D image oftissues via manipulations provided by a user; and a user interface forthe user to adjust parameters of the tool model, wherein said tool modelis displayed on said display dynamically interacting with said 3D imageof tissues according to the adjusted parameters for realisticallysimulating the medical procedure.
 2. The modeling system of claim 1,wherein the user interface to adjust parameters allows modifying astructure of said user tool.
 3. The modeling system of claim 2, whereinsaid modifying a structure of said user tool includes changing a shapeof said user tool.
 4. The modeling system of claim 2, further comprisingan interface for outputting specifications for said user tool asmodified that are generated by the system.
 5. The modeling system ofclaim 4, wherein said specifications are provided in a format forallowing a manufacture of tools to manufacture an actual user tooldesigned for use in performing an actual surgery on the patient.
 6. Themodeling system of claim 1, wherein said medical images of theparticular patient include an image of an aneurysm, and wherein saiddynamic 3D image includes an image of the aneurysm, and further whereinsaid user tool includes an aneurysm clip applier for applying ananeurysm clip model for dynamically interacting with the image oftissues.
 7. The modeling system of claim 1, further comprising: a memoryfor storing a library of a plurality of models of different implants tothe user; and a user interface for selecting one implant model from saidplurality of models for use with said user tool model for dynamicallyinteracting with said 3D image of tissues.
 8. The modeling system ofclaim 1, wherein said dynamically interacting is provided by executingalgorithms adapted for providing segmentation and soft tissue simulationand collision detection for calculating orientation interaction anddetecting interaction between the user tool and the 3D image of tissues.9. The modeling system of claim 1, wherein said system is adapted fordisplaying an image of at least a portion of the head of the patient onsaid display, said image of at least a portion of the head being derivedfrom said medical images of the particular patient.
 10. The modelingsystem of claim 1, wherein the user interface to adjust modelingparameters also allows the user to adjust the mechanical properties ofthe 3D image of tissues.
 11. A modeling system for performing asimulation of an aneurysm clipping surgery, comprising: a database forstoring patient tissue image information that are taken from, or derivedfrom, medical images of a particular patient, said image informationincluding an aneurysm; said database also for storing standardcharacteristics of said tissue; a display; an image generator forgenerating a dynamic 3D image of tissues of the particular patient fordisplay on said display, said generating utilizing said patient imageinformation such that said dynamic 3D image of tissues realisticallyrepresents corresponding actual tissues of the particular patient andincludes at least one blood vessel showing an image of the aneurysm in3D; a tool interface for connecting to a real aneurysm clip applieradapted for use with the modeling system; a user tool generator forgenerating a tool model of an aneurysm clip applier for dynamicallyinteracting with said image of the aneurysm via manipulations of thereal aneurysm clip applier by a user; and a user interface for the userto adjust modeling parameters of the aneurysm clip model, wherein saidaneurysm clip model is displayed on said display dynamically interactingwith said image of the aneurysm according to the adjusted modelingparameters for realistically simulating the aneurysm clipping surgery.12. The modeling system of claim 11, wherein the user interface toadjust modeling parameters of the aneurysm clip model is for modifying adimension or orientation of said aneurysm clip model.
 13. The modelingsystem of claim 12, wherein said modifying a structure of said aneurysmclip model includes changing a shape of said aneurysm clip model. 14.The modeling system of claim 11, further comprising an interface foroutputting specifications for an actual aneurysm clip based on saidaneurysm clip model as modified, said specifications being generated bythe system.
 15. The modeling system of claim 14, wherein saidspecifications are provided in a format for allowing a manufacture ofaneurysm clips to manufacture the actual aneurysm clip for use inperforming an actual aneurysm clipping surgery on the patient.
 16. Themodeling system of claim 11, further comprising: a memory for storing alibrary of a plurality of models of different aneurysm clips to theuser; and a user interface for selecting one aneurysm clip model fromsaid plurality of models for use as said aneurysm clip model fordynamically interacting with said image of the aneurysm.
 17. Themodeling system of claim 11, wherein said dynamically interacting isprovided by executing algorithms adapted for providing segmentation andsoft tissue simulation and collision detection for calculatingorientation interaction and detecting interaction between the model ofthe aneurysm clip and the aneurysm image.
 18. The modeling system ofclaim 11, wherein said system is adapted for displaying an image of atleast a portion of the head of the patient, said image of at least aportion of the head being derived from said medical images of theparticular patient.
 19. The modeling system of claim 11, wherein theuser interface to adjust modeling parameters is also for the user toadjust the mechanical properties of the image of the aneurysm.
 20. Amodeling system for performing a simulation of an aneurysm clippingsurgery, comprising: a database for storing patient tissue imageinformation that are taken from, or derived from, medical images of aparticular patient, said image information including an aneurysm; saiddatabase also for storing standard characteristics of said tissue; adisplay; an image generator for generating a dynamic 3D image of tissuesof the particular patient for display on said display, said generatingutilizing said patient image information such that said dynamic 3D imageof tissues realistically represents corresponding actual tissues of theparticular patient and includes at least one blood vessel showing animage of the aneurysm in 3D; a memory for storing a library of aplurality of models of different aneurysm clips to a user; a userinterface for selecting one aneurysm clip model from said plurality ofmodels for use as said aneurysm clip model for dynamically interactingwith said image of the aneurysm; a tool interface for connecting to areal aneurysm clip applier adapted for use with the modeling system; aninterface for the user to adjust the mechanical properties of the imageof the aneurysm; a user tool generator for generating a model of ananeurysm clip applier; said user tool generator also for generating theselected aneurysm clip model shown dynamically interacting with saidmodel of the aneurysm clip applier applying the model of the aneurysmclip to the image of the aneurysm via manipulations of the real aneurysmclip applier by the user; and an interface for the user to adjustmechanical properties of the model of the aneurysm clip and/or the imageof the aneurysm, wherein said aneurysm clip model is displayed on saiddisplay dynamically interacting with said image of the aneurysm based onthe mechanical properties as adjusted by the user for realisticallysimulating the aneurysm clipping surgery.
 21. A modeling system forperforming a simulation of an implant surgery, comprising: a databasefor storing patient tissue image information that are taken from, orderived from, medical images of a particular patient; said database alsofor storing standard characteristics of said tissue; a display; an imagegenerator for generating a dynamic 3D image of the patient's anatomicalstructure of tissues including blood vessels and bones of the particularpatient for display on said display, said generating utilizing saidpatient image information such that said dynamic 3D image of tissuesrealistically represents corresponding actual tissues of the particularpatient in 3D; a memory for storing a library of a plurality of modelsof different implants to a user; a user interface for selecting oneimplant model from said plurality of models for dynamically interactingwith said image of anatomical structure; an interface for the user toadjust the mechanical properties of the image of the anatomicalstructure; a surgical tool generator for generating a model of asurgical tool; said surgical tool generator also for generating theselected implant model shown dynamically interacting with the image ofthe anatomical structure via manipulations provided by a user; aninterface for the user to manipulate the model of the surgical tool forinteracting with image of the anatomical structure; an interface for theuser to adjust mechanical properties of the model of the surgical tooland/or the implant model and/or the image of the anatomical structure,wherein said implant model is displayed on said display dynamicallyinteracting with said image of the tissues based on the mechanicalproperties as adjusted by the user for realistically simulating thesurgery.
 22. The modeling system of claim 21, wherein said surgery isfor replacing a portion of bone, a bone joint, a knee, or a part of theheart, and wherein said implant is adapted for the particular surgery.